Papers with AI feedback
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment (2024.emnlp-main)
Copied to clipboard
Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong, Qi Liu
| Challenge: | Large vision-language models (LVLMs) are evolving rapidly and require data with human supervision to achieve better alignment. |
| Approach: | They introduce VLFeedback, the first large-scale vision-language feedback dataset . they train Silkie, an LVLM fine-tuned via direct preference optimization . |
| Outcome: | The proposed model outperforms its base model in helpfulness, visual faithfulness, and safety metrics and exhibits enhanced resilience against red-teaming attacks. |
Improving Contrastive Learning of Sentence Embeddings from AI Feedback (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn sentence embeddings with rich semantics are limited due to the discrete nature of natural language. |
| Approach: | They propose to use AI feedback to improve contrastive learning of sentence embeddings by combining human feedback and AI feedback. |
| Outcome: | The proposed method achieves state-of-the-art performance on several semantic textual similarity and transfer learning tasks compared to other unsupervised and supervised contrastive learning methods. |
Check Your Work: Structured Checklist Feedback for Improving Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have been driven by verifiable feedback in deterministic domains like mathematics and code. |
| Approach: | They propose to decompose granular, prompt-specific checklists into a scalar reward and use them to transform them into skalar rewards. |
| Outcome: | The proposed approach yields an 11.8% win-rate improvement on AlpacaEval 2.0 using Qwen3-8B, outperforming holistic reward models and existing checklist baselines. |